Codecov Report

❌ Patch coverage is 90.55556% with 34 lines in your changes missing coverage. Please review.

... and 3 files with indirect coverage changes

I remember that BP is equivalent to simple update (PhysRevResearch.3.023073), although I haven't looked at the details (such as whether the bond messages are guaranteed to be real diagonal, and thus already represented by SUWeight). We may use this equivalence to avoid introducing redundant structs, or generalize the existing simple update algorithm.

BTW, YASTN people are also working on BP, and with NNN Hamiltonian support (in one of the branches) as well.

I looked into trying to reuse some of that functionality, and at least for the BPEnv struct there really are substantial differences: note how there are 2 messages per edge, one going in each direction, which are connected between bra and ket, instead of between the virtual edges of the PEPS. That's why I ended up modeling this more like the CTMRGEnv, where I'm effectively storing a CTMRG environment with chi = 1 trivial legs. (I'll try add a converter/promoter from BPEnv to CTMRGEnv to at some point.)

I do agree that it would be nice to join the simple update part, but for now I was actually mostly interested in affecting the gauge of a PEPS, and we can look into extending this to actually do updates in the future.

It's nice to hear that YASTN also are finding this helpful! I don't think adding NNN support would be too hard, the contractions with these rank 1 environments are a lot more manageable in general.
Something I would like to look into at some point is improving the accuracy with loop corrections, see https://arxiv.org/pdf/2409.03108

Thanks for getting this started! I'd be happy to join in on this since I was recently talking to Bram about PEPS optimization, how well it might perform and how that might be related to gauge freedoms. Specifically, I was looking at optimization runs that get stuck in linesearching at some point - e.g. when choosing a CTMRG environment dimension that is too low. Around these iterations where the optimizer gets stuck I computed cuts through the energy landscape along the current gradient directions and I was getting all sorts of weird curves with cusps or discontinuities (inspired by Appendix B of the periodic PEPS paper: https://arxiv.org/pdf/2411.12731).

For example, a Heisenberg model at $D=3$ and $\chi=4$ will look like this ($i$ being the LBFGS iteration, $g$ the gradient at iteration $i$):

At $D=3$ and $\chi=8$ it looks a bit more wild:

In these cases you can really see that a low environment dimension will first get you smooth convex functions that become more closely centered around $\alpha=0$ and then cusps and discontinuities may appear. In these cases the linesearching algorithms of course run into trouble and eventually may spit out unphysical optimization steps. Also, this process of breaking down might stretch over many LBFGS iterations.

I seemed that gauging by symmetrizing the PEPS spatially did improve the energy landscapes but I haven't properly investigated that. In any case, I would be quite interested to see how BP gauging could improve optimization convergence and perhaps stabilize more complicated optimization runs.

Something I would like to look into at some point is improving the accuracy with loop corrections, see https://arxiv.org/pdf/2409.03108

I also have wanted to try it out for quite a while, and see how it can systemically improve over SU.

To suggest some tests for the current PR, we can first check that if the bond weights produced by SU (with identity gates) are consistent with the BP message tensors.

@pbrehmer these are some really cool plots! This was indeed one of the primary use-cases I had in mind, so it would be cool to see how this alters these gradient plots.

As a side note, at such low bond dimensions I would also be slightly cautious about picking bond dimensions that necessarily break the symmetry, for example in a U1 case with charge conjugation I would expect that you might need to check for each D whether or not it gives a valid combination. This is motivated by similar problems for MPS simulations, where certain bond dimensions don't work because they have to cut inbetween multiplets.

@pbrehmer Indeed, we have seen this a lot for periodic PEPS, and pushing $$\chi$$ and imposing some rotation and mirror symmetries help to stabilize the optimization and avoid these discontinuous points. However, there are many cases, beyond ising and heisenberg models, when imposing spatial symmetries leads to higher energies compared let's say with just SU (or with AD fixed-point one before the optimization arrives at "bad" point :) )

However, there are many cases, beyond ising and heisenberg models, when imposing spatial symmetries leads to higher energies compared let's say with just SU (or with AD fixed-point one before the optimization arrives at "bad" point :) )

Yes I have also noticed that recently in some cases! Good to know that that seems to be consistent with your findings :)

Resolved the src/PEPSKit.jl conflict if anyone wants to pick this back up

Your PR no longer requires formatting changes. Thank you for your contribution!

The tests now pass for all cases I can think of (bosons vs fermions, positive vs arbitrary messages, and standard vs random virtual PEPS arrows). For fermions I introduced some twists to make things work at some seemingly unexpected positions. I'll try to justify them later.

• It might be more convenient in the long run to have the messages follow the "canonical arrow" convention, such that north and east we get top <- bot and south and west we get bot <- top

Now I use this axis order convention suggested by @lkdvos earlier, without additional adjoints.

Sorry I commented on the subcomments before I saw your full comment here.
I am guessing that the choice to omit the adjoints is actually exactly what is causing these twists to appear, as an adjoint would transpose without actually having to introduce a minus sign, but I'm not 100% sure about that. In any case, I would also be happy with the current choices, as that is more close to how we described the algorithms in the MPS case in the fermionic methods paper, although I still think in that case the twists are better placed in the contractions, rather than absorbed in the messages.

I cannot approve since this is my own PR, but I think most of the TODOs and comments have been addressed, so I would be happy to get this one merged.

The only remaining question is whether to absorb the twists into the messages. @lkdvos do you think it's OK to address this later if needed? @kshyatt Maybe you can have a final review and approve.

Yes, I think this PR has lived for too long and it will be easier to review that change in a separate PR

I added some comments to justify the twists used in the gauging part. Now I feel that it is better to keep the current design, so we don't need to worry about manually putting back the twists during BP iterations or evaluation of expectation values (which can become a bit annoying IMO: now we need to add twists only in two places in the code, but otherwise there will be many more).